Locality/Fairness-Aware Job Scheduling in Distributed Stream Processing Engines
نویسندگان
چکیده
منابع مشابه
Benchmarking Distributed Stream Processing Engines
Over the last years, stream data processing has been gaining atten— tion both in industry and in academia due to its wide range of appli— cations. To fulfill the need for scalable and efficient stream analyt— ics, numerous open source stream data processing systems (SDPSs) have been developed, with high throughput and low latency being their key performance targets. In this paper, we propose a ...
متن کاملReplica-Aware Job Scheduling in Distributed Systems
This paper proposes an effective replica-aware scheduling algorithm for independent jobs in Grid and distributed systems. The proposed algorithm considers not only the execution time of jobs but also the location and transfer time of data and data replica that these jobs require. We propose a cost model to estimate the starting time and earliest completion time of a job and its associated data ...
متن کاملProgressive Recovery of Correlated Failures in Distributed Stream Processing Engines
Correlated failures in large-scale clusters have significant effects on systems’ availability, especially for streaming data applications that run continuously and require low processing latency. Most stateof-the-art distributed stream processing engines (DSPEs) adopt a blocking recovery paradigm, which, upon correlated failure, would block the progress of recovery until sufficient new resource...
متن کاملData-bandwidth-aware Job Scheduling Techniques in Distributed Systems
This paper introduces techniques in scheduling jobs on a master/workers platform where the bandwidth is shared by all workers. The jobs are independent and each job requires a fixed amount of bandwidth to download input data before execution. The master can communicate with multiple workers simultaneously, provided that the bandwidth used by the master and the workers do not exceed their bandwi...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Electronics
سال: 2020
ISSN: 2079-9292
DOI: 10.3390/electronics9111857